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Abstract 

A new concept named nonsymmetric entropy which generalizes the 
concepts of Boltzman's entropy and shannon's entropy, was introduced. 
Maximal nonsymmetric entropy principle was proven. Some important 
distribution laws were derived naturally from maximal nonsymmetric en- 
tropy principle. 

PACS: 89.75.-k 



1 introduction 

Entropy which measures the uncertain degree of information is an important 
basic concept in statistic physics and information theory. In Ref. [1] and its 
references, entropy is discussed from many aspects. In present paper, I give a 
new entropy named nonsymmetric entropy which measures the average value 
of the auxihary and probabihty two kinds of information to an event. I prove 
the corresponding maximal nonsymmetric entropy principle. Some interesting 
distribution laws can be derived naturally from this principle. 

2 Basic conceptions and results 

Firstly we give some concepts in the following. 

Definition l:The auxiliary information of an event Xi is defined by 

A(xO = -ln(A), (1) 
where Pi is auxiliary information parameter. 
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Definition 2: Total information of an event Xi is 



It{xi) = I{xi) + A{xi) = - lnp{i) -\nf3i = - ln(Ap(i)). (2) 
Definition 3: we define a function 

Sm{p{l),--- ,p{m)) = -Y,Pii)HPiP{i}), (3) 

i=l 

wliere (3i > 0,i — 1, ■ ■ ■ ,m, are nonsymmetric parameters. We call the function 
Sm the nonsymmetric entropy. 

Remark 1: if we take /3i = 1, we get the Shannon's entropy, thus nonsym- 
metric entropy generalizes the concept of Shannon's entropy. 

It is obvious that nonsymmetric entropy measures the expect information of 
the total information of all events. Because we consider auxiliary information 
so that wc can describe event in a more right way. At the same time, we can use 
nonsymmetric parameters Pi to derive some important distribution laws which 
include Zipf's law. In particular, if we take 

A = i", (4) 
then the corresponding nonsymmetric entropy becomes 

m 

S = -Y,p{i)ln(i'^p{i)), (5) 

i=l 

we can call it Zipf's entropy. We have the following result: 

Theorem 1: If p{i) satisfies the following Zipf's distribution law 

then the Zipf's entropy takes the maximum. 

Corollary 1. For Zipf's law, we have Sm+i > Sm, that is, the nonsymmetric 
entropy is increase as m increasing. 

From the above theorem it is easy to see that the Zipf's law can be derived 
naturally from nonsymmetric entropy under some special auxiliary parameters. 
We don't prove this theorem at the present time, in fact we have the following 
more general result. 

Theorem 2: If {p{l), ■ • ■ ,p{m)} satisfies the following distribution 

M«) = r, ^In 1 , (7) 
Pi Z^i=l /3; 

then the nonsymmetric entropy Sm takes the maximum 

S„ = -lnp(l) = ln^-. (8) 
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Proof: Instituting p{in) = 1 — p{l) — • • • — p(m — 1) into the Eq.Q and 
setting its differential to zero yields 

opW p(l) p{m~l)) 

that is, 

p(l) + --- + (l + |^)pW + ---+p(m-l) = l, z = l,...,m-l. (10) 
Solving the equations system (jlOp . we obtain 

1 i = l,---,m, (11) 

where p(l) = ^^^i Denote = (ajj)fcxfc,ay = ^ + since detA^ = 
, where the hat means the corresponding item disap- 



p(m) *-lp(l)...p(i)...p(fe) 

pear, so from a^j = - dpfifdp(j) = IfiJ + ^^^^ ^^"^ matrix A 

(aij)(m-i)x(m-i) is a positive defined matrix. Thus the distribution p{i) 



maximize the nonsymmetric entropy. The proof is completed. 



Remark 2: If we take f3i = i", then we have 

1 P(l) / ^ 

Pi^) = ^^;^-^^ i=l,---,m, (12) 

in particular, we take a ~ 1, this is just the Zipf's law in linguist. If take 
Pi = {"i + 7)", then we give Mandelbrot's law. If we take other values of 
we will give other distribution law. Thus the key is to choose suitable auxiliary 
information parameters Pi, this is a problem need to study deeply. 

Remark 3: Using maximal nonsymmetric entropy principle in Section 3, 
we can give a simple proof for theorem 2. 

We consider the continuous case in the following. 

Definition 4: For continuous case, nonsymmetric entropy is defined 

S{p) = -J p(x) \n{P(x)p{x)}dx, (13) 

where P{x) is auxiliary information parameter function, p{x) is probability den- 
sity of event. 

In order to solve maximal nonsymmetric entropy distribution, we can use 
variant method. Under some constrains conditions, we use lagrange multiply 
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method to do this thing. We give several example in the following to illustrate 
our method. 

Theorem 2: Assume J xp{x)dx = /x, we then its maximal nonsymmetric 
entropy distribution is 

Po{x) = exp(l - Ai - X2X), (14) 

where Ai and A2 satisfy two constrain conditions / exp(l — Ai — A2a;)da; = 1 
and / exp(l — Ai — A2a;)da; = /U. 
Proof: Make a auxiliary functional 



F{p,Xi,X2) = - J p{x)lii{f3{x)p(x)}dx+Xi{J pix)dx-l)+X2iJ xp{x)dx)-p). 

(15) 

We have 

SF = j {Ai + A2a; - ln{p{x)p{x)) - l}Sp{x)dx, (16) 
form 5F = 0, we solve out as follows: 

/5o(a;) = ■^^exp(l - Ai - A2X), (17) 
where Ai and A2 satisfy two constrain conditions / po{x)dx = 1 and / xpo{x)dx = 

Theorem 3: Assume / xp{x)dx = p, and J x'^p{x)dx = cr^, we then its 
maximal nonsymmetric entropy distribution is 

Po{x) = — ^ exp(l - Ai - A2a; - Asa;^), (18) 

where Ai, A2 and A3 satisfy three constrain conditions / po{x)dx = 1 , / xpo{x)dx - 
p and / x'^po(x)dx = a"^ . 

Proof: it is similar with the proof of theorem 2. 

3 Maximal nonsymmetric entropy principle 

We generalize the maximal entropy principle in information theory to the case 
of nonsymmetric entropy. 

Definition 5: Denote A be a class of probability density functions, if po G A, 
such that 

S{po) = max{S{p) : p S A}, (19) 

then pQ is called maximal nonsymmetric entropy distribution, and S'(po) maxi- 
mal nonsymmetric entropy. 

Theorem 4 (mELximal nonsymmetric entropy principle): A is a fixed 
class of probability density functions, if there exists po € A such that 



- j p{x) \n{(3{x)pQ{x)}dx = So (20) 
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is a constant which is irrelative to p{x) for every p, then Pq{x) is maximal 
nonsymmetric entropy distribution, and S{po) = Sq is maximal nonsymmetric 
entropy. For discrete case, this theorem is also right. 
Proof: for arbitrary p £ A, we have 

S{p) = - / p{x)ln{(3{x)p{x)}dx = - f p{x)\n{p{x)po{x)-^}dx 



Poix) 

= j p{x)\n{f3{x)pQ{x)}dx — j p{x)\n — r^dx 



Poix) 

< - I p(x) \n{l3(x)po{x)}dx = 5o, (21) 



then 5*0 is maximal nonsymmetric entropy. Since S{po) = Sq, so po{x) is maxi- 
mal nonsymmetric entropy distribution. The proof is completed. 
Corollary 2: If J j^^dx < oo, we have 

= ^ (22) 

is maximal nonsymmetric entropy distribution, where C = r 1 ^ ■ 

If we take P{x) = x", a > 1, and assume the arrange of random variable X 
is (k, +cxd), then maximal nonsymmetric entropy distribution is 

Poix) = ^x'-", (23) 
a — 1 

it is just the power law distribution in continuous case. If there are some con- 
strains we will get other distributions similar with them in theorem 2 and theo- 
rem 3. On the other hand, we can easily use the maximal nonsymmetric entropy 
principle to give new proof s to theorems 2 and 3. 



4 Discussions 

The above results suggest that the nonsymmetric entropy is a rather funda- 
mental concept that will play an important role in some fields. Perhaps the 
meaning of the nonsymmetric entropy needs a reasonable explanation. It is 
different with Shannon's entropy S = —Y^^iPi'i)^np{i) in some aspects. For 
example, if pij) = 1 and others zeroes, then 5 = 0, but Sm — — ln/3j, this 
implies that there exist some kind of uncertainty in some superficial reliable 
events under the nonsymmetric entropy. Other deep meanings of nonsymmetric 
entropy need more studies. Since the important roles of Boltzman's entropy and 
Shannon's entropy in thermodynamics and information theory respectively, we 
hope that maximal nonsymmetric entropy principle can play a suitable role in 
corresponding fields. 

For example, Zipf's law ( 2 ) which states that the frequency of a word decays 
as a power law of its rank, is regarded as a basic hypothesis with no need for 



5 



explanation in recent models of the evolution of syntactic communication([3]). 
As an empirical law, Zipf's law is the most fundamental fact in quantitative 
linguistics, its meaning is still an open problem which has been tried to explain 
from several aspects of its origins([ll[51[ni[71[H]- It is necessary to find a suitable 
mechanism for Zipf's law. In this paper, Zipf's law is derived naturally by max- 
imizing the nonsymmetric entropy when auxiliary parameter take some special 
values. It is at least need to consider seriously the meaning of those results. I 
will continue to study the theory and applications of nonsymmetric entropy. 
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